It has long been recognized that the rendition of people is one of the most important aspects of color rendition. Consequently, a significant amount of research has been done in the field of the optimum color and tone reproduction of skin tones (D. L. MacAdam, Proc. I.R.E., 42, 1954, 166; R. W. G. Hunt, I. T. Pitt, L. M. Winter, “The Preferred Reproduction of Blue Sky, Green Grass and Caucasian Skin in Colour Photography”, J. Photogr. Science 22, 144,1974; M. Yamamoto, Y -H Lim, X. Wei, M. Inui, H. Kobayashi, “On the Preferred Flesh Colour in Japan, China and South Korea”, The Imaging Science Journal, 51, 163, 2003). This literature indicates that preferences for the rendition of skin tones may differ by geographic region, ethnic and cultural background and skin type. (Also see the discussion of light and dark skin tones in U.S. Pat. No. 6,396,599 B1, to Patton et. al., entitled “Method and Apparatus for Modifying Portions of an Image in Accordance with Colorimetric Parameters (hereafter “Patton”).
Traditionally preferences were addressed by offering different films and sometimes different photographic papers for different geographic markets. Examples are Kodak Professional Portra films and Kodak Ultima 100, a professional film marketed in India. With the advent of digital capture and digital photofinishing it has become possible to address these preferences by customizing a suitable set of image processing parameters.
U.S. Pat. No. 5,300,381, to Keelan et. al., entitled “Customizing a Digital Camera based on Demographic Factors” describes a digital camera customized for skin color and redeye based on demographic factors. Regional customization can be implemented at camera manufacturing, as part of the camera firmware, or by providing different camera menus by region. Regional optimization of skin tone reproduction is performed using region-specific matrix coefficients to convert camera RGB values to Standard CCIR 709 primaries.
U.S. Published Patent Application 2003/0202194 A1, filed by Torigoe, entitled “Image Processing Apparatus and Information Processing Apparatus, and Method Therefore”, (hereafter “Torigoe”) addresses geographic preferences in relation to digital mini labs. Torigoe describes a color managed photofinishing system with multiple input and output devices, referred to as a color conversion apparatus. A storage unit contains output profiles for multiple geographic regions. Two different methods of implementing regional preferences are disclosed. In the first case, the operator accesses a user interface, which displays a triangle with 3 different names of regions, for example, America, Europe, and Asia. A preference anywhere inside the triangle can be selected, and the system will generate the appropriate output by blending the stored regional output profiles. The second method of customizing the output is to select preferences for the reproduction of important memory colors, for example, skin, sky, and foliage, and neutral colors. The user interface has a slider that lets the user select the degree of modification between the standard rendition and the preferred rendition. Color and tone manipulations include selective hue rotations and saturation adjustments of skin tones and brightness and color balance adjustments by region. Torigoe focuses on selective manipulations of the color and brightness of skin tones without special considerations regarding overall tone reproduction or smoothness of skin tone reproduction.
U.S. Pat. No. 6,633,410 B1, to Narushima, entitled “Printer for Printing Visually Equivalent Images Relative to Displayed Images” (hereafter “Narushima”) addresses geographic preferences in relation to photofinishing kiosks. Narushima discloses a color-managed printing system, which contains a processing section for “desirable reproduction”. This module addresses the observation that the preferred reproduction of common memory colors such as skin, foliage, and blue sky may not be the same as accurate calorimetric reproduction, both in terms of color specification as well as tone scale and image structure characteristics. Narushima describes a method for determining the preferred reproduction of various objects by using a judging method that may involve studying the flesh color of a large number of people. These preferences are addressed by selective color manipulations in an approximately perceptually uniform color space, e.g. CIELAB. Schematic diagrams of selectively shifting and or contracting colors in CIELAB are shown. The system allows variable selection and processing methods to determine if a color has to be moved. Such methods may include pattern recognition or analyzing the image to find if data are in a predetermined range. More global manipulations of color are also possible. Narushima teaches that the sequence of processing and the image processing parameters in the desired reproduction section can be customized for the intended user at the time of shipment.
Patton describes a system where the customer and/or lab can manually or automatically select the desired skin tone characteristics. Skin tones are selectively identified in scanned images based on at least one colorimetric parameter. The skin tones are then automatically modified towards the preferred position, which can be different depending on whether a dark or a light skin tone was identified. Different modifications for dark and light skin tones in a single image are possible. Skin tone modifications are achieved by color and/or density balance shifts, but preferably by employing a 3D LUT, which selectively modifies skin tones. While Patton identifies contrast of the face as one of the parameters that can be customized, the issue of implementing a global tone reproduction curve optimized for the rendition of people with a wide range of skin tones is not addressed. Likewise, no considerations are given towards obtaining a good texture of skin in the reproduction.
Other inventions use face detection algorithms to treat images containing people differently from scenic pictures without people. One example is given by U.S. Published Patent Application No. US2003/0035578 A1, filed by Dupin and Luo, entitled “Method for Processing Digital Image to Adjust Brightness”, (hereafter “Dupin”) which describes a method for obtaining improved lightness balance for images containing skin or faces. The image is initially balanced using known scene balance methods, followed by an analysis to detect skin colored pixels that creates a skin probability map using an adaptive thresholding method derived from the gradient of the skin probability map. From the detected skin pixels, a brightness adjustment is calculated from the center weighted average of the skin colored pixels and predetermined constants for typical skin pixel brightness values and general algorithm tuning. Dupin does not teach or offer any direction concerning use or adjustment of the algorithm for regional or cultural preferences for skin brightness levels.
The above approaches have the shortcoming of ignoring the importance of structure and texture in skin tone reproduction. The relationship between the visually perceived densities of objects in an image scene reproduction compared to those in the original scene is a critical aspect of image reproduction. Traditionally, conventional silver-halide based photographic systems have produce a well known non-linear, “S”-shaped relationship between the viewed print density (in the case of a print system) vs. scene exposure, such as shown by “The Reproduction of Colour” by Dr. R. W. G. Hunt, Fountain Press, England, Fourth ed., (hereafter “Hunt”), see p.54. While this type of curve may provide “snappy” reproductions of certain types of scenes, it tends to create a harsh look of peoples faces. With the advent of digital imaging it is possible to create preferred tonal reproductions from any capture source, e.g., photographic films, digital cameras, on a large collection of output media and devices, e.g., photographic, thermal or inkjet paper and any type of electronic display devices, e.g., television, computer displays, personal picture viewing devices and electronic paper.
As a general rule, a smooth appearance of skin tones is preferred. The appropriate texture of skin tones can be achieved using sophisticated image processing algorithms, for example, as disclosed in U.S. Published Patent Application No. US2003/0223622 A1, filed by Simon et al., entitled “Method and System for Enhancing Portrait Images”. The approach is complex and processing intensive.
A smooth appearance of skin tones can also be accomplished by much simpler means of selecting an appropriate tone reproduction for a particular situation. U.S. Pat. No. 5,528,339, to Buhr et al., entitled “Color Image Reproduction of Scenes with Color Enhancement and Preferential Tone Mapping”; and U.S. Pat. No. 5,390,036, to Buhr et al., entitled “Color Image Reproduction of Scenes with Preferential Tone Mapping” disclose use of a family of tone reproduction curves optimized for skin tone reproduction. In U.S. Pat. No. 5,528,339, viewed densities on the print were defined as a function of the densities of the original scene. Limits were provided for the slope of this curve, in particular at medium scene densities, which are representative of skin tones and mid-gray tones. These slope limits are lower than what is currently used in most color reproduction systems. While such low slopes provide a smooth appearance of skin tones, it has been determined that the image, including the face, often looks too flat, with shadows having a tendency to look gray and relatively harsh transitions towards highlights.
It is well known to those skilled in the art that the perception of color includes other important attributes in addition to lightness, as addressed by the tone reproduction curve. Attributes that are frequently used in connection with perceptually uniform color spaces are lightness, chroma or saturation, and hue. Preferences for skin tone reproduction can be expressed in those terms, in particular using the CIELAB system (Yamamoto and K. Töpfer, R. E. Cookingham, “The Quantitative Aspects of Color Rendering for Memory Colors”, Proceedings of IS&T's 2000 PICS Conference, Portland, Oreg., p. 94.). Individual and regional preferences for lightness, hue and saturation can be addressed by applying overall color and density shifts to the image or by selectively shifting certain regions of color space, e.g., important memory colors, such as skin tones. This approach is disclosed in Torigoe, Narushima, and EP 1139653 A2, filed by Buhr et al., entitled “Color Image Reproduction of Scenes with Preferential Color Mapping”.
These approaches of selective color manipulations of skin tones ignore the wider issue of the preferred rendition of people, which also includes hair reproduction and reproduction of whites. White reproduction is important for rendering the background of eyes, teeth and for retaining good detail in clothing. Moreover, they tend ignore the importance of a smooth texture and overall appearance of human faces in image reproductions of people.
It would thus be desirable to provide methods and systems, which have an improved smooth rendering of skin tones and an improved rendering of people overall.